Papers with teacher forcing
BrainECHO: Semantic Brain Signal Decoding through Vector-Quantized Spectrogram Reconstruction for Whisper-Enhanced Text Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Current EEG/MEG-to-text decoding systems rely on teacher-forcing methods . pre-trained large language models are over-dominant in decoding text from brain activity . |
| Approach: | They propose a framework that employs decoupled representation learning to achieve state-of-the-art performance on EEG and MEG datasets. |
| Outcome: | The proposed framework achieves state-of-the-art performance on EEG and MEG datasets. |
Can Latent Alignments Improve Autoregressive Machine Translation? (2021.naacl-main)
Copied to clipboard
| Challenge: | Latent alignment objectives improve non-autoregressive models, but can they improve autoregressive ones? e.g., we show that latent alignments are incompatible with teacher forcing. |
| Approach: | They propose latent alignment objectives that use a dynamic program to comb the space of monotonic alignments between the "gold" target sequence and token probabilities the model predicts. |
| Outcome: | The proposed models are incompatible with teacher forcing, the authors show . they show that latent alignment objectives reduce misalignments and focus on original error . |
Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation models are usually based on attention-based encoder-decoder frameworks. |
| Approach: | They introduce a seer decoder into the encoder-decoder framework during training . they force the conventional decoded decodes to simulate the behavior of the seer . |
| Outcome: | The proposed method outperforms baselines on Chinese, English and German translation tasks. |
Exploring Question-Specific Rewards for Generating Deep Questions (2020.coling-main)
Copied to clipboard
| Challenge: | Recent question generation approaches use the sequence-to-sequence framework to optimize the log likelihood of ground-truth questions using teacher forcing. |
| Approach: | They propose to optimize for QG-specific objectives via reinforcement learning to improve question quality. |
| Outcome: | The proposed model improves the fluency, relevance, and answerability of generated questions. |
Evaluating Rewards for Question Generation Models (N19-1)
Copied to clipboard
| Challenge: | Recent approaches to question generation have used modifications to a Seq2Seq architecture inspired by advances in machine translation. |
| Approach: | They propose to use a Seq2Seq architecture to train models to generate one-step-ahead predictions, but at test time, the model is asked to generate a whole sequence, causing errors to propagate through the generation process. |
| Outcome: | The proposed model is trained to generate a plausible question, conditioned on an input document and answer span within that document. |
QuantileMark: A Message-Symmetric Multi-bit Watermark for LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | a number of large language models (LLMs) require multi-bit watermarking to ensure provenance. |
| Approach: | They propose a multi-bit watermark that embeds messages within a continuous cumulative probability interval. |
| Outcome: | The proposed watermark breaks message symmetry in low-entropy decoding, showing it can be used for verification and quality verification. |
Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Exposure bias is a central problem for auto-regressive language models (LM) it is believed that teacher forcing would cause test-time generation to be incrementally distorted due to the training-generation discrepancy. |
| Approach: | They propose to quantify the impact of exposure bias in quality, diversity, consistency and consistency by using ground-truth data prefixes instead of prefix generated by the model. |
| Outcome: | The proposed model performs better when the training-generation discrepancy is removed . the model is more robust and self-recovery ability is shown to counter exposure bias. |
The Mirage of Model Editing: Revisiting Evaluation in the Wild (2025.acl-long)
Copied to clipboard
| Challenge: | despite near-perfect results, effectiveness of model editing in real-world applications remains unclear. |
| Approach: | They propose QAEdit and WILD to better reflect real-world use of model editing . they propose a benchmark aligned with widely used question answering datasets and a task-agnostic evaluation framework . |
| Outcome: | The proposed QAEdit benchmark and WILD evaluation framework show that current models perform worse than previously reported. |
Semformer: Transformer Language Models with Semantic Planning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Neural language models (LLMs) employ teacher forcing to predict tokens based on preceding ground truth tokens. |
| Approach: | They propose a method for training a Transformer language model that explicitly models the semantic planning of response. |
| Outcome: | The proposed method exhibits near-perfect performance and mitigates shortcut learning. |
CPC-GRPO: Answer-Free Reinforcement Learning with Cross-Prompt Consensus Rewards (2026.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards is a popular post-training tool for large language models, but relies on a ground-truth answer or external verifier, which limits applicability and increases cost. |
| Approach: | They propose an answer-free training objective that derives rewards solely from the model’s own probabilities by exploiting prompt paraphrases as multiple semantic views of the same intent. |
| Outcome: | The proposed objective derives rewards solely from the model’s own probabilities by exploiting prompt paraphrases as multiple semantic views of the same intent. |